Environmental driver comfort feedback for autonomous vehicle

ABSTRACT

Described herein is an autonomous vehicle comprising a sensor and a driver notification module in communication with the sensor. The driver notification module may be configured to receive first information related to an upcoming event; determine an action for an autonomous vehicle based, at least in part, on the first information; determine one or more characteristics of the action based, at least in part on the first information and a current status of the autonomous vehicle; determine a notification based, at least in part, on the action and the one or more characteristics; and deliver the notification within the autonomous vehicle.

TECHNICAL FIELD

The subject matter described herein relates in general to autonomousvehicles and, more particularly, to environmental driver comfortfeedback for autonomous vehicles.

BACKGROUND

Autonomous vehicles react to the driving environment around them.Autonomous vehicles recognize traffic signs, traffic lights, and othersurrounding objects. Sometimes the autonomous vehicle may stop, changelanes, swerve, or take other actions based on surrounding objects. Thedriver and/or occupants of the vehicle may be alarmed by sudden changesin the vehicle's status, if they are unaware of the upcoming change.

SUMMARY

In an embodiment herein, a driver notification module is described, thedriver notification module comprising a memory; and a processor coupledto the memory. The processor may be configured to: receive firstinformation related to an upcoming event; determine an action for anautonomous vehicle based, at least in part, on the first information;determine one or more characteristics of the action based, at least inpart on the first information and a current status of the autonomousvehicle; determine a notification based, at least in part, on the actionand the one or more characteristics; and deliver the notification withinthe autonomous vehicle.

In another embodiment herein, method for driver notification isdescribed. The method may comprise receiving, by a driver notificationmodule, first information related to an upcoming event; determining, bythe driver notification module, an action for an autonomous vehiclebased, at least in part, on the first information; determining, by thedriver notification module, one or more characteristics of the actionbased, at least in part on the first information and a current status ofthe autonomous vehicle; determining, by the driver notification module,a notification based, at least in part, on the action and the one ormore characteristics; and delivering, by the driver notification module,the notification within the autonomous vehicle.

In another embodiment herein, an autonomous vehicle is described. Theautonomous vehicle may comprise a sensor; and a driver notificationmodule in communication with the sensor. The driver notification modulemay be configured to receive first information related to an upcomingevent; determine an action for an autonomous vehicle based, at least inpart, on the first information; determine one or more characteristics ofthe action based, at least in part on the first information and acurrent status of the autonomous vehicle; determine a notificationbased, at least in part, on the action and the one or morecharacteristics; and deliver the notification within the autonomousvehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of an embodiment of a system for environmentaldriver comfort feedback.

FIG. 2 is a diagram of an embodiment of a view from the interior of avehicle.

FIG. 3 is a flow diagram of an embodiment of a method for environmentaldriver comfort feedback.

FIG. 4 is a block diagram of an embodiment of a system for environmentaldriver comfort feedback for autonomous vehicles.

DETAILED DESCRIPTION

Described herein is a system and method for environmental driver comfortfeedback for autonomous vehicles. When an autonomous vehicle isoperating autonomously, the vehicle may be configured to recognizevehicle traffic, signs, traffic lights, road hazards, and other upcomingevents. The vehicle operating autonomously may include any action takenby the vehicle that does not require driver intervention. Upcomingevents may include any situation or object that the vehicle mayencounter in the future. The autonomous vehicle may be configured totake an action based on the upcoming events. A driver and/or occupantsof the vehicle may become uncomfortable if the vehicle takes an actionunbeknownst to the driver and/or occupants. For example, if a vehicle isapproaching a red light, the vehicle may recognize the red light andabruptly stop. The driver and/or occupants of the vehicle may beuncomfortable because they are unaware of the reason for the stop.

An autonomous vehicle may be configured with a driver notificationsystem. The driver notification system may detect an upcoming event anddetermine a corresponding action for the upcoming event. The drivernotification system may also determine characteristics of the action tobe taken. For example, the driver notification may notify the driverthat a red light has been detected and that the vehicle will be stoppingshortly. The notification may be audible or visual or both. Thenotification may place the driver at ease when the action is taken.

FIG. 1 is a diagram of an embodiment of a system 100 for environmentaldriver comfort feedback. The system 100 may comprise a vehicle 110, anevent and action database 130 and a geography data store 135. In someembodiments, the event and action database 130 and the geography datastore 135 may collocated or may be stored in separate locations. Vehicle110 may communicate with the event and action database 130 and thegeography data store 135 via the internet 120 using an antenna 115.Antenna 115 may be configured to communicate with the internet 120 usingwireless communication. The event and action database 130 and thegeography data store 135 may be part of cloud based storage or someother data aggregation storage solution. The vehicle 110 may have asensor 140 for sensing the area around the vehicle 110. Sensor 140 maybe radar, LIDAR, a camera, and/or any other sensor for sensing the areaaround the vehicle 110. The vehicle 110 may also have a drivernotification module 150. Driver notification module 150 may be astand-alone module or may be part of another system within the vehicle110.

The event and action database 130 may be a parameterized database thatassociates events that vehicle 110 may encounter to actions vehicle 110may take in response to the event. The actions may be based on the typeand/or severity of the event. The actions may be associated with anotification to be provided to driver and/or occupants of the vehicle.The notification may notify driver and/or occupants of the upcomingevent and the action taken in response to the event. The notificationmay be a visual notification, an audio notification, or both.

The geography data store 135 may contain information related togeography vehicle 110 may encounter. The geography data store 135 maycontain road maps, topography maps, traffic maps, traffic statistics,and other information that may be used by vehicle 110. Geography datastore 135 may contain locations of traffic lights and other trafficcontrol devices. Geography data 135 and sensor 140 may be used alone orin combination to determine upcoming events that a driver may benotified of.

FIG. 2 is a diagram of an embodiment of a view 200 from the interior ofvehicle 110. Notifications of upcoming events and actions may beprovided to the vehicle driver and/or occupants audibly and/or visually.In an embodiment an upcoming event 210 may be detected. In this case,the upcoming event 210 may be stopped traffic. Driver notificationmodule 150 may determine the upcoming event 210 and a correspondingaction. The driver notification module may then determinecharacteristics of the action and a phrase or alert that may be providedto driver and/or occupants of the vehicle. The phrase or alert may beprovided visually via a heads-up-display (HUD) 220 and/or an in-dashdisplay 230, and/or audibly via speaker 240. The in-dash display 230 mayinclude one or more of a navigation system, entertainment system,instrument cluster, or some other display in the vehicle visible by thedriver and/or occupants of the vehicle. In an embodiment, the HUD mayhighlight the upcoming event. For example, the HUD may project a boxaround an upcoming red light.

FIG. 3 is a flow diagram of an embodiment of a method 300 forenvironmental driver comfort feedback. Method 300 may begin at block 310when an upcoming event is detected by a vehicle, e.g., vehicle 110.Upcoming events may be any situation that vehicle may encounter thatcauses the vehicle to change its current operating status. Examples ofupcoming events include, but are not limited to: traffic signals,traffic hazards, bicyclists, stopped traffic, potholes, etc.

The upcoming event may be detected using data from a sensor, e.g.,sensor 140, data from a geography data store, e.g., geography data store135, other data sources, or any combination thereof. For example, sensor140 may comprise a camera and may capture an image of a bicyclist on theshoulder of the road ahead of the vehicle. As another example, drivernotification module 150, may interact with geography data store 135 todetermine a sharp curve in the upcoming road. As yet another example,sensor 140 may detect a pothole in the road and the location of thedetected pothole may be correlated with a location of the pothole fromgeography data store 135.

After an upcoming event is detected, the method may continue at block320 where an action that corresponds to the event may be detected. Theaction may be any action that causes the vehicle to change its currentoperating status. Examples of actions include, but are not limited to:deceleration, swerving, stopping lane changing, accelerating, activatinga turn signal, etc.

Driver notification module 150 may interact with the event and actionsdatabase 130 to determine actions that correspond to the upcoming event.For example, an action associated with the bicyclist on the shoulder ofthe road ahead of the vehicle may include a lane shift. As anotherexample, an action associated with the sharp curve in the upcoming roadmay include deceleration of the vehicle. As yet another example, anaction associated with the pothole in the road may include the vehicleswerving to avoid the pothole.

After an action is determined, the method may continue at block 330where characteristics of the action may be determined. Thecharacteristics of the action may be determined based upon the upcomingevent and/or the determined action. Examples of characteristics of theaction include, but are not limited to: intensity of the action,duration of the action, time until the action occurs, etc.

Driver notification module 150 may interact with the event and actionsdatabase 130 to determine characteristics of the action. For example,characteristics of a lane shift may include identifying which lane thevehicle will shift to and/or when the shift will occur. As anotherexample, characteristics of deceleration may include identifying howmuch velocity will decrease as a result of the deceleration, theduration of the deceleration, and/or identifying when the decelerationwill begin. As yet another example, characteristics of swerving mayinclude direction of the swerve, intensity of the swerve, and/or whenthe swerve will occur.

After an action and characteristics of the action have been determined,the method may continue at block 340 where the driver and/or occupantsof the vehicle may be notified. The driver notification module 150 maycreate a notification by retrieving a notification previously stored ina database, e.g., event and actions database 130. The drivernotification module 150 may create the notification using a text tospeech generator and the previously determined action andcharacteristics of the action. Creating the notification may includeother actions taken by the driver notification module 150 to provide thevehicle occupants with information related to the action and/orcharacteristics of the action. The notification may include the upcomingevent, the action, and/or characteristics of the action. Thenotification may be provided audibly and/or visually. The notificationmay be provided via any combination of HUD 220, in-dash display 230, orspeaker 240.

Driver notification module 150 may interact with vehicle systems toprovide the notification. For example, the driver notification module150 may cause the notification “bicyclist ahead, the vehicle will changeto the left lane in 35 seconds” to be displayed and played to the driverand/or vehicle occupants. As another example, the driver notificationmodule 150 may cause the notification “sharp curve ahead, the vehiclewill be decelerating to 25 miles per hour” to be displayed and played tothe driver and/or vehicle occupants. As yet another example, the drivernotification module 150 may cause the notification “pothole ahead, thevehicle will be swerving to the right in 400 feet” to be displayed andplayed to the driver and/or vehicle occupants.

After the driver and/or occupants of the vehicle are notified, themethod may continue at step 350, where the vehicle takes the actiondescribed in the notification. In some embodiments, this step isoptional, e.g., the driver may take control of the vehicle or thevehicle may determine autonomous driving is no longer possible basedupon the event.

FIG. 4 illustrates an example of a system 400 that includes a processor410 suitable for implementing one or more embodiments disclosed herein.The processor 410 may control the overall operation of the device. In anembodiment, the system 400 may be configured as a driver notificationmodule 150. System 400 may be part of vehicle 110 comprising the drivernotification module 150.

In addition to the processor 410 (which may be referred to as a centralprocessor unit or CPU), the system 400 might include networkconnectivity devices 420, random access memory (RAM) 430, read onlymemory (ROM) 440, secondary storage 450, and input/output (I/O) devices460. These components might communicate with one another via a bus 470.In some cases, some of these components may not be present or may becombined in various combinations with one another or with othercomponents not shown. These components might be located in a singlephysical entity or in more than one physical entity. Any actionsdescribed herein as being taken by the processor 410 might be taken bythe processor 410 alone or by the processor 410 in conjunction with oneor more components shown or not shown in the drawing, such as a digitalsignal processor (DSP) 480. Although the DSP 480 is shown as a separatecomponent, the DSP 480 might be incorporated into the processor 410.

The processor 410 executes instructions, codes, computer programs, orscripts that it might access from the network connectivity devices 420,RAM 430, ROM 440, or secondary storage 450 (which might include variousdisk-based systems such as hard disk, floppy disk, or optical disk).While only one CPU 410 is shown, multiple processors may be present.Thus, while instructions may be discussed as being executed by aprocessor, the instructions may be executed simultaneously, serially, orotherwise by one or multiple processors. The processor 410 may beimplemented as one or more CPU chips and may be a hardware devicecapable of executing computer instructions. In an embodiment, theprocessor 410 may be configured to execute instructions that cause adriver notification module to perform the method of FIG. 3.

The network connectivity devices 420 may take the form of modems, modembanks, Ethernet devices, universal serial bus (USB) interface devices,serial interfaces, token ring devices, fiber distributed data interface(FDDI) devices, wireless local area network (WLAN) devices, radiotransceiver devices such as code division multiple access (CDMA)devices, global system for mobile communications (GSM) radio transceiverdevices, universal mobile telecommunications system (UMTS) radiotransceiver devices, long term evolution (LTE) radio transceiverdevices, worldwide interoperability for microwave access (WiMAX)devices, and/or other well-known devices for connecting to networks.These network connectivity devices 420 may enable the processor 410 tocommunicate with the Internet or one or more telecommunications networksor other networks from which the processor 410 might receive informationor to which the processor 410 might output information. The networkconnectivity devices 420 might also include one or more transceivercomponents 425 capable of transmitting and/or receiving data wirelessly.

The RAM 430 might be used to store volatile data and perhaps to storeinstructions that are executed by the processor 410. The ROM 440 is anon-volatile memory device that typically has a smaller memory capacitythan the memory capacity of the secondary storage 450. ROM 440 might beused to store instructions and perhaps data that are read duringexecution of the instructions. Access to both RAM 430 and ROM 440 istypically faster than to secondary storage 450. The secondary storage450 is typically comprised of one or more disk drives or tape drives andmight be used for non-volatile storage of data or as an over-flow datastorage device if RAM 430 is not large enough to hold all working data.Secondary storage 450 may be used to store programs that are loaded intoRAM 430 when such programs are selected for execution.

The I/O devices 460 may include liquid crystal displays (LCDs), touchscreen displays, keyboards, keypads, switches, dials, mice, track balls,voice recognizers, card readers, paper tape readers, printers, videomonitors, or other well-known input/output devices. Also, thetransceiver 425 might be considered to be a component of the I/O devices460 instead of or in addition to being a component of the networkconnectivity devices 420.

Detailed embodiments are disclosed herein; however, it is to beunderstood that the disclosed embodiments are intended only as examples.Therefore, specific structural and functional details disclosed hereinare not to be interpreted as limiting, but merely as a basis for theclaims and as a representative basis for teaching one skilled in the artto variously employ the aspects herein in virtually any appropriatelydetailed structure. Further, the terms and phrases used herein are notintended to be limiting but rather to provide an understandabledescription of possible implementations. Various embodiments are shownin FIGS. 1-7, but the embodiments are not limited to the illustratedstructure or application.

It will be appreciated that for simplicity and clarity of illustration,where appropriate, reference numerals have been repeated among thedifferent figures to indicate corresponding or analogous elements. Inaddition, numerous specific details are set forth in order to provide athorough understanding of the embodiments described herein. However, itwill be understood by those of ordinary skill in the art that theembodiments described herein can be practiced without these specificdetails.

The flowcharts and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments. In this regard, each block in the flowcharts or blockdiagrams may represent a module, segment, or portion of code, whichcomprises one or more executable instructions for implementing thespecified logical function(s). It should also be noted that, in somealternative implementations, the functions noted in the block may occurout of the order noted in the figures. For example, two blocks shown insuccession may, in fact, be executed substantially concurrently, or theblocks may sometimes be executed in the reverse order, depending uponthe functionality involved.

The systems, components and/or processes described above can be realizedin hardware or a combination of hardware and software and can berealized in a centralized fashion in one processing system or in adistributed fashion where different elements are spread across severalinterconnected processing systems. Any kind of processing system orother apparatus adapted for carrying out the methods described herein issuited. A typical combination of hardware and software can be aprocessing system with computer-usable program code that, when beingloaded and executed, controls the processing system such that it carriesout the methods described herein. The systems, components and/orprocesses also can be embedded in a computer-readable storage, such as acomputer program product or other data programs storage device, readableby a machine, tangibly embodying a program of instructions executable bythe machine to perform methods and processes described herein. Theseelements also can be embedded in an application product which comprisesall the features enabling the implementation of the methods describedherein and, which when loaded in a processing system, is able to carryout these methods.

Furthermore, arrangements described herein may take the form of acomputer program product embodied in one or more computer-readable mediahaving computer-readable program code embodied or embedded, e.g.,stored, thereon. Any combination of one or more computer-readable mediamay be utilized. The computer-readable medium may be a computer-readablesignal medium or a computer-readable storage medium. The phrase“computer-readable storage medium” means a non-transitory storagemedium.

What is claimed is:
 1. A driver notification module comprising: amemory; and a processor coupled to the memory, the processor configuredto: receive first information related to an upcoming event; determine anaction for an autonomous vehicle based, at least in part, on the firstinformation; determine one or more characteristics of the action based,at least in part on the first information and a current status of theautonomous vehicle; create a notification based, at least in part, onthe action and the one or more characteristics; and deliver thenotification within the autonomous vehicle.
 2. The driver notificationmodule of claim 1, wherein the first information is received from one ofa sensor of the autonomous vehicle or a geography data store.
 3. Thedriver notification module of claim 1, wherein the notification isdelivered via at least one of an audio device and a video device.
 4. Thedriver notification module of claim 1, wherein the processor is furtherconfigured to: receive second information related to the upcoming event,wherein the first information is received from a sensor of theautonomous vehicle and the second information is received from ageography data store; determine the action for the autonomous vehiclebased, at least in part, on the first information and the secondinformation; and determine the one or more characteristics of the actionbased, at least in part on the first information, the secondinformation, and a current status of the autonomous vehicle.
 5. Thedriver notification module of claim 1, wherein at least a portion of thenotification is retrieved from a database, the database configured tomap the one or more characteristics to the portion of the notification.6. The driver notification module of claim 5, wherein the database isfurther configured to map the one or more characteristics to the action.7. The driver notification module of claim 1, wherein the action isrelated to a change in a current operating status of the autonomousvehicle.
 8. A method for driver notification, the method comprising:receiving, by a driver notification module, first information related toan upcoming event; determining, by the driver notification module, anaction for an autonomous vehicle based, at least in part, on the firstinformation; determining, by the driver notification module, one or morecharacteristics of the action based, at least in part on the firstinformation and a current status of the autonomous vehicle; creating, bythe driver notification module, a notification based, at least in part,on the action and the one or more characteristics; and delivering, bythe driver notification module, the notification within the autonomousvehicle.
 9. The method of claim 8, wherein receiving the firstinformation comprises receiving the first information from one of asensor of the autonomous vehicle or a geography data store.
 10. Themethod of claim 9, wherein delivering the notification comprisesdelivering the notification via at least one of an audio device and avideo device.
 11. The method of claim 8 further comprising: receivingsecond information related to the upcoming event, wherein the firstinformation is received from a sensor of the autonomous vehicle and thesecond information is received from a geography data store; determiningthe action for the autonomous vehicle based, at least in part, on thefirst information and the second information; and determining the one ormore characteristics of the action based, at least in part on the firstinformation, the second information, and a current status of theautonomous vehicle.
 12. The method of claim 8 further comprising:retrieving at least a portion of the notification from a database, thedatabase configured to map the one or more characteristics to theportion of the notification.
 13. The method of claim 12, wherein thedatabase is further configured to map the one or more characteristics tothe action.
 14. The method of claim 8, wherein the action is related toa change in a current operating status of the autonomous vehicle.
 15. Anautonomous vehicle comprising: a sensor; and a driver notificationmodule in communication with the sensor, the driver notification moduleconfigured to: receive first information related to an upcoming event;determine an action for an autonomous vehicle based, at least in part,on the first information; determine one or more characteristics of theaction based, at least in part on the first information and a currentstatus of the autonomous vehicle; create a notification based, at leastin part, on the action and the one or more characteristics; and deliverthe notification within the autonomous vehicle.
 16. The autonomousvehicle of claim 15, wherein the first information is received from oneof the sensor or a geography data store.
 17. The autonomous vehicle ofclaim 15 further comprising: an audio device; and a video device,wherein the notification is delivered via at least one of an audiodevice and a video device.
 18. The autonomous vehicle of claim 15,wherein the driver notification module is further configured to: receivesecond information related to the upcoming event, wherein the firstinformation is received from the sensor and the second information isreceived from a geography data store; determine the action for theautonomous vehicle based, at least in part, on the first information andthe second information; and determine the one or more characteristics ofthe action based, at least in part on the first information, the secondinformation, and a current status of the autonomous vehicle.
 19. Theautonomous vehicle of claim 15, wherein at least a portion of thenotification is retrieved from a database, the database configured to:map the one or more characteristics to the portion of the notification;and map the one or more characteristics to the action.
 20. Theautonomous vehicle of claim 15, wherein the action is related to achange in a current operating status of the autonomous vehicle.